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Abstract 

This paper describes properties of the tanh function from the standpoint of its use in robust signal 
processing and statistical estimation. The construction of circuits that perform tanh is very natural 
in analog integrated circuit technology. Therefore, this transform is well-suited for use in circuits 
that perform computations on (noisy) signals, as well as its use in statistical data analyses. 
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1 Introduction 


/gX q ~ x '\ 

The hyperbolic tangent function, tanh (a:) = , provides a useful transformation in situations 

where it is intended to reduce the effect of outlying values of a variable. The tanh transform maps 
the real line onto [—1,1]. For small values of x, tanh(x) ~ x. Thus, with appropriate scaling, tanh 
can be used to provide a linear transformation for input values in the neighborhood of “expected” 
values while reducing the effect of values that are outside the expected range. Figure 1 shows a plot 
of tanh(x) vs. x. 


behavior of tanh 



x 


Figure 1: Behavior of tanh 

An application where this transformation plays a key role is the construction of robust analog 
VLSI systems. C. Mead [ ] was first to point out the usefulness of tanh as a robust transformation 
and to show that the transform derives naturally from the exponential behavior of MOS transistors 
when operating in the subthreshold regime. As shown in Mead [ ] pg. 70, a simple transconductance 
amplifier that exhibits a tanh transfer function can be constructed using five transistors. 
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2 STATISTICAL ANALYSIS 


2 Statistical Analysis 


It is useful to show the behavior of this transform as applied to input values that are treated as a 
random variable with a given probability distribution. This transformation is derived below. 


2.1 Probability Density Function of a r.v. transformed by tanh 


If Y = g(X) is a single-valued and continuous transformation of a random variable X, then Y is a 
random variable whose distribution can be evaluated as follows (Fisz [2] pg. 36). Take h(y) to be 
the inverse of g{x). Then, 

rx 2 ry 2 

P{x 1 < X < x 2 ) = / f(x)dx = / f[h{y)]ti(y)dy. (1) 

Jxi Jyi 

For h'(y) > 0 the integral on the right gives P(yi <Y< y 2 ). If h'(y ) < 0, then y 2 < yi, and 

ryi ryi 

P(x 1 <X<x 2 ) = - f[h(y)]h'(y)dy = / f[h(y)]\ti (y)\dy 

Jy 2 Jy 2 (2) 

= P(y 2 < Y < yi) . 


Thus, f[h(y)]\h' (y)\ gives the density of the transformed random variable Y. 

The function tanh(x) is one-to-one and has a continuous first derivative. Therefore, the density 
of y = tanh(x) is given by f[y] = f[h(y)]\h'(y)\, where: 


h(y) = tanh 1 (y) = - In 

,// \ dy 1 

h {y) = dx = ■ 


y + 1 
y- 1 


( 3 ) 


2.2 Probability Density of tanh of a N(/i,a) random variable 

Using the above result, tanh of a random variable with density N(/r, a) has density: 

r 1 2 


f[y) 


1 1 

(1 — y 2 ) ^27tct 2 )2 





Note that 

\ ln (ihiy) = tanh_1 (2/) = y + \y 3 + \y 5 + ■ ■ ■ ■ 

From this expansion it is easy to see that for y <C 1 the density is approximately N(/r, a). 


( 4 ) 

( 5 ) 


2.2 Probability Density of tanh of a N(fr , o ) random variable 
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The tanh transformation tends to yield a density with local maxima near —1 and 1. For any 
symmetric distribution the position of the larger of the two maxima has the same sign as the mean 
of the original distribution. Thus, tanh can be thought of as a transformation that tends to cause 
signals to assume one of two states, depending on the sign of the original signal relative to the 
center of its range. The transformed distribution, for zero mean, becomes significantly bimodal for 
values of <r 2 greater than 1. 


Figure 2a: transformed density 
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Figure 2b: transformed density 
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Figure 2c: transformed density 
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Figure 2: Transformed Densities 


Figures 2a-c show the behavior of the transformation for a selection of values of /r and o. The 
Figures show how the position of the maximum of the transformed density varies as a function of 
the position of the maximum of the original Normal distribution and as a function of <r 2 . Figures 
2a, 2b and 2c show this relationship for selected values of /r. Each plot shows 4 values of <x 2 . For <r 2 
very small the relationship is very nearly linear. But, as o 2 increases the relationship approaches 
an exponential function. A small change in the mean of the Normal distribution results in the 
maximum of the transformed distribution rapidly approaching 1 or —1. 
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3 Numerical Evaluation of ta n h 

tanh can be numerically evaluated from tanh(x) = e eX f e e - x which reduces to tanh(.x) = ■ Thus 

the computation for an input vector x[l : n] and transformed output xt[ 1 : n] is: 

z = exp( 2*x); 
xt = (z— 1 )./(z + 1); 

The main work in the code above is the n evaluations of exp(). While this is not likely to be a 
substantial part of the computation to be performed on the input vector, it is very much less efficient 
than the direct implementation in an analog VLSI circuit. 

4 Circuit Implementation 

The feature that makes tanh particularly useful in analog circuit design is the fact that it can be 
implemented very simply. The following figure shows a basic differential pair , which can be used to 
compose many useful circuits (See Mead [1] pg. 67). 


r % 1 

V 


9 , 


Differential Pair 

Figure 3: Differential Pair 

The current difference out of this circuit is given by: 

h-I 2 = h tanh < Vl ~ V ^ ( 7 ) 

where k = relates the surface potential if to the gate voltage V g . 

References 

1. C. A. Mead. Analog VLSI and Neural Systems. Addison- Wesley, Reading, MA, USA, 1989. 

2. M. Fisz. Probability Theory and Mathematical Statistics. John Wiley & Sons., London &; New 
York, 1963. 


